Skip to content

dispatch Series[datetime64] ops to DatetimeIndex #19024

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 4, 2018

Conversation

jbrockmendel
Copy link
Member

This is the culmination of a bunch of recent work.

If merged, this will subsume #18960 and will obviate parts of #18964. This will also fix (but not test, so we'll leave the issue open for now) the lack of overflow checks #12534. Also in a follow-up we'll be able to remove a bunch of _TimeOp.

@jreback
Copy link
Contributor

jreback commented Jan 1, 2018

rebase and push again, fixed some hanging by Travis CI

@@ -744,6 +744,13 @@ def wrapper(left, right, name=name, na_op=na_op):
return NotImplemented

left, right = _align_method_SERIES(left, right)
if is_datetime64_dtype(left) or is_datetime64tz_dtype(left):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and until / unless you want to limit / remove _TimeOp (which is actually ok with me). then this doesn't belong here as I have commented before.

You are welcome to put it in _TimeOp for now or rip out TimeOp.

@jreback
Copy link
Contributor

jreback commented Jan 1, 2018

I think it will be much cleaner / better to have 1 PR which addresses any/all of TimeOp issues. You seem to want to change it, which is totally fine. But rather than just hack around, let's blow it away.

@codecov
Copy link

codecov bot commented Jan 1, 2018

Codecov Report

Merging #19024 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #19024      +/-   ##
==========================================
+ Coverage   91.56%   91.58%   +0.01%     
==========================================
  Files         150      150              
  Lines       48942    48897      -45     
==========================================
- Hits        44816    44784      -32     
+ Misses       4126     4113      -13
Flag Coverage Δ
#multiple 89.95% <100%> (+0.01%) ⬆️
#single 41.74% <60%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/ops.py 91.91% <100%> (+1.49%) ⬆️
pandas/core/computation/expressions.py 93.27% <0%> (-1.69%) ⬇️
pandas/util/testing.py 84.95% <0%> (+0.21%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04beec7...4efb27a. Read the comment docs.

@jbrockmendel
Copy link
Member Author

Yes, I very much want to get rid of _TimeOp. The strategy so far has been to first address the various ways in which _TimeOp behaves differently from DatetimeIndex/TimedeltaIndex, that way we make sure a) we have appropriate tests for each bug that gets fixed, b) appropriate WhatsNew notes, and c) check that these differences are in fact unintentional.

You are welcome to put it in _TimeOp for now or rip out TimeOp.

For reasons discussed elsewhere, putting it in _TimeOp is not a viable option long-term. What we can do is I can amend this PR to rip out the is_datetime_lhs cases, leaving only the is_timedelta_lhs cases. We're a few steps away from being able to rip those out, but we're getting there.

@jreback
Copy link
Contributor

jreback commented Jan 1, 2018

that would ls be fine

@@ -208,6 +208,9 @@ Other API Changes
- In :func:`read_excel`, the ``comment`` argument is now exposed as a named parameter (:issue:`18735`)
- Rearranged the order of keyword arguments in :func:`read_excel()` to align with :func:`read_csv()` (:issue:`16672`)
- The options ``html.border`` and ``mode.use_inf_as_null`` were deprecated in prior versions, these will now show ``FutureWarning`` rather than a ``DeprecationWarning`` (:issue:`19003`)
- Subtracting ``NaT`` from a :class:`Series` with ``dtype='datetime64[ns]'`` returns a ``Series`` with ``dtype='timedelta64[ns]'`` instead of ``dtype='datetime64[ns]'``(:issue:`18808`)
- Operations between a :class:`Series` with dtype ``dtype='datetime64[ns]'`` and a :class:`PeriodIndex` will correctly raises ``TypeError`` (:issue:`18850`)
- Subtraction of :class:`Series` with timezone-aware ``dtype='datetime64[ns]'`` will mis-matched timezones will raise ``TypeError`` instead of ``ValueError`` (issue:`18817`) dtypewith mis-matched timezones will now raise a ``TypeError`` instead of a ``ValueError`` (:issue:`18817`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like some text got duped in the last entry

@@ -107,7 +107,7 @@ def test_shift(self):
# incompat tz
s2 = Series(date_range('2000-01-01 09:00:00', periods=5,
tz='CET'), name='foo')
pytest.raises(ValueError, lambda: s - s2)
pytest.raises(TypeError, lambda: s - s2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this in the whatsnew notes API changed section?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@jreback jreback added this to the 0.23.0 milestone Jan 4, 2018
@jreback jreback merged commit 3198b9d into pandas-dev:master Jan 4, 2018
@jreback
Copy link
Contributor

jreback commented Jan 4, 2018

see how easy that was! removing code is fun!

anyhow obviously want to come back and clean this up even more, and definitly add some comments on what is non-obvious in the code (e.g. where datetimes are dispatched to DTI)

@jbrockmendel jbrockmendel deleted the dt_ops branch January 23, 2018 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Datetime Datetime data dtype Timedelta Timedelta data type
Projects
None yet
3 participants